Simulations of estimating causal effects using undersmoothed HAL
Seraphina Shi
2023-05-10
Simulation 1
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-5 + W + 2.25A -0.5WA)]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-3.778440 Min. :0.0000 Min. :0.000
## 1st Qu.:-0.673429 1st Qu.:0.5824 1st Qu.:0.000
## Median :-0.005279 Median :2.0233 Median :0.000
## Mean :-0.000091 Mean :2.1235 Mean :0.465
## 3rd Qu.: 0.673443 3rd Qu.:3.4201 3rd Qu.:1.000
## Max. : 4.061050 Max. :5.0000 Max. :1.000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.432 2.718 2.690 4.133 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.8792 2.3269 2.3381 3.6968 5.0000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.3424 1.7209 1.9081 3.1282 5.0000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.325 1.557 2.586 5.000
n = 200
## The average lambda of CV-HAL: 0.0142 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0044 (= 0.3129 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0118, 0.0115, 0.0115, 0.0116, 0.0118, 0.0121, 0.0123, 0.0124, 0.0124, 0.0124, 0.0124 (= [0.8446, 0.8299, 0.8286, 0.829, 0.8406, 0.8601, 0.8737, 0.8797, 0.8794, 0.8814, 0.8846] * lambda_CV## The average fitting time for CV-HAL: 4.9150 seconds## The average fitting time for globally undersmoothed HAL: 7.3811 seconds## The average fitting time for locally undersmoothed HAL: 31.2348 seconds
n = 500
## The average lambda of CV-HAL: 0.0056 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0017 (= 0.2954 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0041, 0.004, 0.004, 0.004, 0.004, 0.0041, 0.0042, 0.0043, 0.0044, 0.0044, 0.0044 (= [0.7114, 0.6953, 0.6941, 0.6953, 0.6956, 0.7073, 0.7306, 0.7531, 0.7661, 0.7696, 0.7751] * lambda_CV## The average fitting time for CV-HAL: 5.3269 seconds## The average fitting time for globally undersmoothed HAL: 10.3145 seconds## The average fitting time for locally undersmoothed HAL: 57.3556 seconds
n = 1000
## The average lambda of CV-HAL: 0.0029 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0008 (= 0.2740 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0018, 0.0017, 0.0016, 0.0016, 0.0016, 0.0017, 0.0018, 0.0019, 0.0019, 0.0019, 0.002 (= [0.6072, 0.5747, 0.567, 0.56, 0.5592, 0.5821, 0.6146, 0.6403, 0.6668, 0.6747, 0.6785] * lambda_CV## The average fitting time for CV-HAL: 5.8674 seconds## The average fitting time for globally undersmoothed HAL: 11.6748 seconds## The average fitting time for locally undersmoothed HAL: 63.8782 seconds
Simulation 2
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 + 2W + 5sin(A^{1.5}) + 2WA)]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-4.023488 Min. :0.0000 Min. :0.0000
## 1st Qu.:-0.683524 1st Qu.:0.5998 1st Qu.:0.0000
## Median : 0.002623 Median :2.0083 Median :0.0000
## Mean :-0.002187 Mean :2.1171 Mean :0.0665
## 3rd Qu.: 0.666584 3rd Qu.:3.3949 3rd Qu.:0.0000
## Max. : 3.819893 Max. :5.0000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.408 2.743 2.703 4.078 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.8347 2.2646 2.3032 3.6241 5.0000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.4634 1.7649 1.9340 3.1372 5.0000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.292 1.531 2.534 5.000
n = 200
## The average lambda of CV-HAL: 0.0036 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0016 (= 0.3371 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0036, 0.0035, 0.0034, 0.0034, 0.0034, 0.0034, 0.0034, 0.0034, 0.0035, 0.0034, 0.0034 (= [1, 0.9924, 0.9842, 0.9853, 0.9863, 0.9869, 0.9887, 0.9887, 0.9897, 0.9824, 0.9841] * lambda_CV## The average fitting time for CV-HAL: 4.9046 seconds## The average fitting time for globally undersmoothed HAL: 6.5215 seconds## The average fitting time for locally undersmoothed HAL: 19.8505 seconds
## [1] "proportion of simulations that cannot compute empirical SD:"## 1.2 1.1 1 0.6952 0.4833 0.336 0.2336 0.1624 0.1129 0.0785 0.0546
## 0.740 0.748 0.752 0.774 0.830 0.844 0.838 0.864 0.898 0.902 0.924
## 0.0379 0.0264 0.0183 0.0127 0.0089 0.0062 0.0043 0.003 0.0021 0.0014 0.001
## 0.938 0.950 0.954 0.944 0.958 0.954 0.960 0.952 0.956 0.962 0.966Simulation 3
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 - 3W + 4A + \mathbf{I}(A>2) * 5sin((0.8A)^2 - 2.6) )]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-3.829350 Min. :0.0000 Min. :0.0000
## 1st Qu.:-0.684888 1st Qu.:0.5926 1st Qu.:0.0000
## Median : 0.000602 Median :2.0179 Median :0.0000
## Mean :-0.001718 Mean :2.1279 Mean :0.4387
## 3rd Qu.: 0.678175 3rd Qu.:3.4197 3rd Qu.:1.0000
## Max. : 3.818122 Max. :5.0000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.424 2.819 2.744 4.195 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.911 2.260 2.311 3.645 5.000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.3843 1.8105 1.9431 3.1824 5.0000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.248 1.519 2.555 5.000
n = 200
## The average lambda of CV-HAL: 0.0026 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0009 (= 0.2837 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0024, 0.0024, 0.0024, 0.0024, 0.0025, 0.0025, 0.0025, 0.0024, 0.0025, 0.0025, 0.0026 (= [0.8706, 0.8682, 0.8726, 0.8927, 0.9219, 0.9584, 0.9378, 0.9093, 0.9588, 0.9761, 0.9854] * lambda_CV## The average fitting time for CV-HAL: 3.7293 seconds## The average fitting time for globally undersmoothed HAL: 6.0426 seconds## The average fitting time for locally undersmoothed HAL: 24.0037 seconds
n = 500
## The average lambda of CV-HAL: 0.0009 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0003 (= 0.2800 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 8e-04, 8e-04, 8e-04, 8e-04, 8e-04, 9e-04, 8e-04, 8e-04, 9e-04, 9e-04, 9e-04 (= [0.9227, 0.9227, 0.9237, 0.9206, 0.9183, 0.9397, 0.9147, 0.9201, 0.9661, 0.9833, 0.9895] * lambda_CV## The average fitting time for CV-HAL: 6.1899 seconds## The average fitting time for globally undersmoothed HAL: 14.7461 seconds## The average fitting time for locally undersmoothed HAL: 85.6112 seconds
Simulation 4
Data structure: \(O = (W_1, W_2, W_3, W_4, W_5, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- Structural equations F and endogenous variables:
- \(W \sim N(\mu_W, \Sigma_W)\)
- \(A = bound(0.1W_1 + 0.2W_2 + 0.5W_3 + 0.15W_4 - 0.05W_5 - 0.01W_3W_5 + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-7 -W_1 +
2W_2 - 0.5W_4 - W_1W_3 + 4A + 0.5AW_2)]\)
- where \(\mu_W = \begin{bmatrix}0 \\0 \\5 \\1 \\1 \\\end{bmatrix}\), and \(\Sigma_W = \begin{bmatrix}1&0.8&1.35&0.1&0.03 \\0.8&1&1.2&0.15&0.03 \\1.35&1.2&2.25&0.075&0.225 \\0.1&0.15&0.075&0.25&0.1125 \\0.03&0.03&0.225&0.1125&0.09 \\\end{bmatrix}\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W1 W2 W3 W4
## Min. :-3.575703 Min. :-3.842447 Min. :-0.6526 Min. :-0.9272
## 1st Qu.:-0.673817 1st Qu.:-0.686913 1st Qu.: 3.9847 1st Qu.: 0.6407
## Median :-0.009942 Median :-0.001605 Median : 4.9996 Median : 0.9881
## Mean :-0.007964 Mean :-0.005344 Mean : 5.0056 Mean : 0.9870
## 3rd Qu.: 0.661308 3rd Qu.: 0.658679 3rd Qu.: 6.0171 3rd Qu.: 1.3361
## Max. : 3.637353 Max. : 3.862201 Max. :10.4060 Max. : 2.9160
## W5 A Y
## Min. :-0.3758 Min. :0.000 Min. :0.0000
## 1st Qu.: 0.7711 1st Qu.:1.608 1st Qu.:0.0000
## Median : 0.9939 Median :2.553 Median :1.0000
## Mean : 1.0004 Mean :2.548 Mean :0.6216
## 3rd Qu.: 1.2334 3rd Qu.:3.512 3rd Qu.:1.0000
## Max. : 2.1796 Max. :5.000 Max. :1.0000
Simulation 5
Data structure: \(O = (W_1, W_2, W_3, W_4, W_5, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- Structural equations F and endogenous variables:
- \(W \sim N(\mu_W, \Sigma_W)\)
- \(A = bound(0.1W_1 + 0.2W_2 + 0.5W_3 + 0.15W_4 - 0.05W_5 - 0.01W_3W_5 + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 -W_1 +
2W_2 - 0.5W_4 - 0.5W_1W_3 + 4A + \mathbf{I}(A>2) *5sin((0.8A)^2 -
2.6))]\)
- where \(\mu_W = \begin{bmatrix}0 \\0 \\5 \\1 \\1 \\\end{bmatrix}\), and \(\Sigma_W = \begin{bmatrix}1&0.8&1.35&0.1&0.03 \\0.8&1&1.2&0.15&0.03 \\1.35&1.2&2.25&0.075&0.225 \\0.1&0.15&0.075&0.25&0.1125 \\0.03&0.03&0.225&0.1125&0.09 \\\end{bmatrix}\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W1 W2 W3 W4
## Min. :-4.10305 Min. :-3.992762 Min. :-0.723 Min. :-1.0645
## 1st Qu.:-0.66856 1st Qu.:-0.665766 1st Qu.: 3.993 1st Qu.: 0.6574
## Median : 0.01409 Median : 0.004221 Median : 5.007 Median : 0.9974
## Mean : 0.01345 Mean : 0.004542 Mean : 5.008 Mean : 0.9924
## 3rd Qu.: 0.68434 3rd Qu.: 0.676313 3rd Qu.: 6.014 3rd Qu.: 1.3343
## Max. : 3.37515 Max. : 4.074948 Max. :10.222 Max. : 2.8006
## W5 A Y
## Min. :-0.1632 Min. :0.000 Min. :0.000
## 1st Qu.: 0.7565 1st Qu.:1.604 1st Qu.:0.000
## Median : 0.9909 Median :2.561 Median :0.000
## Mean : 0.9929 Mean :2.554 Mean :0.476
## 3rd Qu.: 1.2283 3rd Qu.:3.509 3rd Qu.:1.000
## Max. : 2.2176 Max. :5.000 Max. :1.000
Simulation 6
Data structure: \(O = (W_1, W_2, W_3, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 1
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Structural equations F and endogenous variables:
- \(W_1 \sim Uniform(o,1)\)
- \(W_2 \sim Bernoulli(\mu=0, \sigma^2 = 2^2)\)
- \(W_3 \sim N(W_1, 0.25*exp(2W_1))\)
- \(A \sim Beta(v(W)\mu(W), v(W)[1-\mu(W)])\)
- \(Y \sim Bernoulli(Q_0(A,W))\)
- where:
- \(v(W) = exp(1 + 2W_1expit(W3))\)
- \(\mu(W) = expit(0.03 - 0.8log(1+W_2) + 0.9exp(W_1)W_2 - 0.4arctan(W_3+2)W_2W_1)\)
- \(\bar{Q}_0(A,W) = expit(-2 + 1.5A + 5A^3 - 2.5W_1 + 0.5AW_2 - log(A)W_1W_2 + 0.5A^{3/4}W_1W_3)\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in (0,1])\), the causal dose-response curve
## W1 W2 W3 A
## Min. :0.0000212 Min. :0.0000 Min. :-3.20646 Min. :0.008172
## 1st Qu.:0.2550912 1st Qu.:0.0000 1st Qu.: 0.07964 1st Qu.:0.483313
## Median :0.5016742 Median :1.0000 Median : 0.80953 Median :0.661615
## Mean :0.5026247 Mean :0.6984 Mean : 0.80890 Mean :0.632873
## 3rd Qu.:0.7530768 3rd Qu.:1.0000 3rd Qu.: 1.53093 3rd Qu.:0.807451
## Max. :0.9999523 Max. :1.0000 Max. : 5.10017 Max. :0.999333
## Y
## Min. :0.000
## 1st Qu.:0.000
## Median :0.000
## Mean :0.483
## 3rd Qu.:1.000
## Max. :1.000
n = 200
## The average lambda of CV-HAL: 0.0090 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0023 (= 0.2632 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.006, 0.0058, 0.0057, 0.0057, 0.0056, 0.0056, 0.0056, 0.0054, 0.0053, 0.0051, 0.0049, 0.0044, 0.0043, 0.0043, 0.0042, 0.0045, 0.0045 (= [0.6913, 0.676, 0.6703, 0.666, 0.6611, 0.6609, 0.6546, 0.6404, 0.6239, 0.603, 0.5776, 0.5095, 0.4942, 0.4902, 0.4755, 0.5031, 0.5107] * lambda_CV## The average fitting time for CV-HAL: 40.4222 seconds## The average fitting time for globally undersmoothed HAL: 46.4000 seconds## The average fitting time for locally undersmoothed HAL: 159.3079 seconds
